{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPyTorch regression with derivative information\n",
    "\n",
    "## Introduction\n",
    "In this notebook, we show how to train a GP regression model in GPyTorch of an unknown function given function value and derivative observations. We consider modeling the function:\n",
    "\n",
    "\\begin{align*}\n",
    "              y &= \\sin(2x) + cos(x) + \\epsilon \\\\\n",
    "  \\frac{dy}{dx} &= 2\\cos(2x) - \\sin(x) + \\epsilon \\\\  \n",
    "       \\epsilon &\\sim \\mathcal{N}(0, 0.5)\n",
    "\\end{align*}\n",
    "\n",
    "using 50 value and derivative observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import gpytorch\n",
    "import math\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the training data\n",
    "We use 50 uniformly distributed points in the interval $[0, 5 \\pi]$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "lb, ub = 0.0, 5*math.pi\n",
    "n = 50\n",
    "\n",
    "train_x = torch.linspace(lb, ub, n).unsqueeze(-1)\n",
    "train_y = torch.stack([\n",
    "    torch.sin(2*train_x) + torch.cos(train_x), \n",
    "    -torch.sin(train_x) + 2*torch.cos(2*train_x)\n",
    "], -1).squeeze(1)\n",
    "\n",
    "train_y += 0.05 * torch.randn(n, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the model\n",
    "A GP prior on the function values implies a multi-output GP prior on the function values and the partial derivatives, see 9.4 in http://www.gaussianprocess.org/gpml/chapters/RW9.pdf for more details. This allows using a `MultitaskMultivariateNormal` and `MultitaskGaussianLikelihood` to train a GP model from both function values and gradients. The resulting RBF kernel that models the covariance between the values and partial derivatives has been implemented in `RBFKernelGrad` and the extension of a constant mean is implemented in `ConstantMeanGrad`.\n",
    "\n",
    "The `RBFKernelGrad` is generally worse conditioned than the `RBFKernel`, so we place a lower bound on the noise parameter to keep the smallest eigenvalues of the kernel matrix away from zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GPModelWithDerivatives(gpytorch.models.ExactGP):\n",
    "    def __init__(self, train_x, train_y, likelihood):\n",
    "        super(GPModelWithDerivatives, self).__init__(train_x, train_y, likelihood)\n",
    "        self.mean_module = gpytorch.means.ConstantMeanGrad()\n",
    "        self.base_kernel = gpytorch.kernels.PolynomialKernelGrad(power=2)\n",
    "        self.covar_module = gpytorch.kernels.ScaleKernel(self.base_kernel)\n",
    "\n",
    "    def forward(self, x):\n",
    "        mean_x = self.mean_module(x)\n",
    "        covar_x = self.covar_module(x)\n",
    "        return gpytorch.distributions.MultitaskMultivariateNormal(mean_x, covar_x)\n",
    "\n",
    "likelihood = gpytorch.likelihoods.MultitaskGaussianLikelihood(num_tasks=2)  # Value + Derivative\n",
    "model = GPModelWithDerivatives(train_x, train_y, likelihood)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the model\n",
    "The model training is similar to training a standard GP regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 1/50 - Loss: 93.724   offset: 0.693   noise: 0.693\n",
      "Iter 2/50 - Loss: 92.825   offset: 0.644   noise: 0.744\n",
      "Iter 3/50 - Loss: 92.090   offset: 0.598   noise: 0.798\n",
      "Iter 4/50 - Loss: 91.469   offset: 0.554   noise: 0.852\n",
      "Iter 5/50 - Loss: 90.969   offset: 0.513   noise: 0.907\n",
      "Iter 6/50 - Loss: 90.599   offset: 0.474   noise: 0.961\n",
      "Iter 7/50 - Loss: 90.282   offset: 0.438   noise: 1.013\n",
      "Iter 8/50 - Loss: 90.035   offset: 0.404   noise: 1.062\n",
      "Iter 9/50 - Loss: 89.839   offset: 0.373   noise: 1.107\n",
      "Iter 10/50 - Loss: 89.683   offset: 0.344   noise: 1.147\n",
      "Iter 11/50 - Loss: 89.550   offset: 0.317   noise: 1.180\n",
      "Iter 12/50 - Loss: 89.449   offset: 0.292   noise: 1.206\n",
      "Iter 13/50 - Loss: 89.356   offset: 0.269   noise: 1.226\n",
      "Iter 14/50 - Loss: 89.259   offset: 0.248   noise: 1.238\n",
      "Iter 15/50 - Loss: 89.167   offset: 0.228   noise: 1.243\n",
      "Iter 16/50 - Loss: 89.065   offset: 0.210   noise: 1.243\n",
      "Iter 17/50 - Loss: 88.967   offset: 0.194   noise: 1.237\n",
      "Iter 18/50 - Loss: 88.869   offset: 0.178   noise: 1.226\n",
      "Iter 19/50 - Loss: 88.776   offset: 0.164   noise: 1.212\n",
      "Iter 20/50 - Loss: 88.686   offset: 0.151   noise: 1.194\n",
      "Iter 21/50 - Loss: 88.599   offset: 0.139   noise: 1.174\n",
      "Iter 22/50 - Loss: 88.512   offset: 0.129   noise: 1.153\n",
      "Iter 23/50 - Loss: 88.433   offset: 0.119   noise: 1.132\n",
      "Iter 24/50 - Loss: 88.361   offset: 0.109   noise: 1.110\n",
      "Iter 25/50 - Loss: 88.299   offset: 0.101   noise: 1.090\n",
      "Iter 26/50 - Loss: 88.242   offset: 0.093   noise: 1.071\n",
      "Iter 27/50 - Loss: 88.192   offset: 0.086   noise: 1.054\n",
      "Iter 28/50 - Loss: 88.142   offset: 0.079   noise: 1.041\n",
      "Iter 29/50 - Loss: 88.098   offset: 0.073   noise: 1.030\n",
      "Iter 30/50 - Loss: 88.058   offset: 0.068   noise: 1.022\n",
      "Iter 31/50 - Loss: 88.012   offset: 0.063   noise: 1.018\n",
      "Iter 32/50 - Loss: 87.968   offset: 0.058   noise: 1.017\n",
      "Iter 33/50 - Loss: 87.922   offset: 0.054   noise: 1.019\n",
      "Iter 34/50 - Loss: 87.876   offset: 0.051   noise: 1.023\n",
      "Iter 35/50 - Loss: 87.832   offset: 0.047   noise: 1.030\n",
      "Iter 36/50 - Loss: 87.789   offset: 0.044   noise: 1.039\n",
      "Iter 37/50 - Loss: 87.746   offset: 0.041   noise: 1.049\n",
      "Iter 38/50 - Loss: 87.707   offset: 0.039   noise: 1.059\n",
      "Iter 39/50 - Loss: 87.672   offset: 0.036   noise: 1.070\n",
      "Iter 40/50 - Loss: 87.639   offset: 0.034   noise: 1.081\n",
      "Iter 41/50 - Loss: 87.608   offset: 0.032   noise: 1.090\n",
      "Iter 42/50 - Loss: 87.580   offset: 0.031   noise: 1.099\n",
      "Iter 43/50 - Loss: 87.553   offset: 0.029   noise: 1.106\n",
      "Iter 44/50 - Loss: 87.526   offset: 0.028   noise: 1.111\n",
      "Iter 45/50 - Loss: 87.500   offset: 0.027   noise: 1.115\n",
      "Iter 46/50 - Loss: 87.475   offset: 0.025   noise: 1.116\n",
      "Iter 47/50 - Loss: 87.449   offset: 0.024   noise: 1.116\n",
      "Iter 48/50 - Loss: 87.424   offset: 0.023   noise: 1.115\n",
      "Iter 49/50 - Loss: 87.399   offset: 0.023   noise: 1.112\n",
      "Iter 50/50 - Loss: 87.375   offset: 0.022   noise: 1.108\n"
     ]
    }
   ],
   "source": [
    "# Find optimal model hyperparameters\n",
    "model.train()\n",
    "likelihood.train()\n",
    "\n",
    "# Use the adam optimizer\n",
    "optimizer = torch.optim.Adam([\n",
    "    {'params': model.parameters()},  # Includes GaussianLikelihood parameters\n",
    "], lr=0.1)\n",
    "\n",
    "# \"Loss\" for GPs - the marginal log likelihood\n",
    "mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)\n",
    "\n",
    "n_iter = 50\n",
    "for i in range(n_iter):\n",
    "    optimizer.zero_grad()\n",
    "    output = model(train_x)\n",
    "    loss = -mll(output, train_y)\n",
    "    loss.backward()\n",
    "    print('Iter %d/%d - Loss: %.3f   offset: %.3f   noise: %.3f' % (\n",
    "        i + 1, n_iter, loss.item(),\n",
    "        model.covar_module.base_kernel.offset.item(),\n",
    "        model.likelihood.noise.item()\n",
    "    ))        \n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making predictions with the model\n",
    "Model predictions are also similar to GP regression with only function values, butwe need more CG iterations to get accurate estimates of the predictive variance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAFwCAYAAABejXgsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xt8XHWdN/DPOXPPJJkkTdOWlrZJAVFuTXtckWUF2+o+FETpli3L+pLHB6muC6KLFfpwW1ywXOyDqMCzla7AS7tPtxVQoa61rYIsVZk2peCCAumN0nZymcllkrme8/xxzkxmJpNkJnM5l/m8X4SZOXMu3yTNb77nd76/3xEURQERERERkdWJegdARERERFQNTHyJiIiIqCYw8SUiIiKimsDEl4iIiIhqAhNfKokgCCsEQXhXEIQHBEFYLQjCNwRB2FeB43QIgrCt3PvNOcYKQRB+VcljEBGZgSAIS/K07WuL2L6kNrsabT7VJoGzOlCptMZpg6Io+7XX31AU5cEy7He1oijbM143KYoSKnW/Uxxzm6IoV1fyGEREZpCnbX8AwK8URdlV4PZFtdl6tPlUe9jjS2UlCEITgP1l2s8nMl53AOgodb9ERDRt/wrggUJWLLbNZptP1cLEl8pthaIouzLLBrRLZQ9oz1cIgvAr7fEbgiAsSW2ovV6hXU6TAEiCIKzO2PcDGeuuzVh30v1q76/WLts1ac+3aY9r812+myT+psw4tcuBK1JfZfspEhEZjKIo3dCS0Txt4WqtDV4tCMI3tE1S7WYh7e+Ebf4E27Mtpmlh4kvlskYQhH8F0AIAOZfC/jX1RFveoj1uB7AGUBNZAN3a8ibtsT912UtrcEPaut/IWLdbK63Iu9+M427XtgkB6AawAUCHoiibAHwx95uZKH4A6wHs195flDpOKpaCf1pERObUpD1mtYVaG9uhPW7KbLMLaX8na/Nzt9fK0dgW07Qw8aVy2aooyhdRWIPTn2fZ0tS2BdQHfzjjON3a64n2m2mb1pvQotWs7dd6BqbaLlMHgCatR7kPagP+CW1AX9OkWxIRmVhOKVtuW4jUexPU5Zba/qa3n+D4bIupIEx8qaxyekpTjV8hdVrvIuMSWub2uWULUJPd1D47ALxaYHj/AbVXoF/rYe5IxavVk+XKF/+rUHsc9gPYBLW041ZFUZYC4OU1IrKytVATTGB8WziVQtrfidr8rO0nOD7bYiqIXe8AyNy0BmoJ1FKHkHZ5KuVV7Yy+CcCKjMEKHdp2KwAs0UbuPqjV0qa23Q61jGE1gF2p4wiC0KEoyq1abRcALNG2XTHBftM9D4qihARB6FcUZb8gCC3aOiug9lIs0RLuJYIgLNEa03Hxa8f6hrY9AHw4J2YiItPLadtTbXcooxQhqy1Mtcep9jOnze6eqv2F2qExUZuftX2+44NtMRWI05kRERERUU1gqQMRERER1QQmvkRERERUE5j4EhEREVFNYOJLRERERDWBiS8RERER1QQmvkRERERUE6o2j+9tt93GedOIyLTuv/9+Yeq1rINtNhGZXb52u6o3sLjnnnuK3iYQCKCtra0C0RSPseRnpFgAY8XDWPIzUizA1PHcfffdVYzGONhml4+RYgGMFQ9jyc9IsQDGiqeQWCZqt1nqQEREREQ1gYkvEREREdWEqpY6EFFxBgcHEQwGEY/HS96XLMsYHBwsQ1SlM1IsQHY8DocDzc3NaGxs1DkqIqqm3t5ehEIhw7RNRm4n9ZYZiyAI8Pl8aG1tLWhbJr5EBtbT04O5c+fC5XJBEEobWxWPx+FwOMoUWWmMFAswFo+iKIhGozh+/DgTX6IaMzAwgPnz58PtdusdCgDjtpNGkBlLMpnE4cOHmfgSWYGiKIZphGuBIAhwu91QFE5oQFRrFEWBzWbTOwwqks1mK6rNZo0vEU2pq6sLXV1dFT9OKBTCM888M+3tqxUnEVElmKWtNTMmvkQWceLECaxYsQInT56c9j66urrwxBNPYPfu3XjiiSfQ3d0NAPD5fNi+fXu5Qp1QU1NT3uPs3r0bs2fPTscTCoWwcuXK9OuU9vZ2PPHEExWPk4hql5Xb2q6uLnzwgx/EM888g2eeeQYbN24c187m093djY0bNxZ07NT+itmmnFjqQGQRGzZswCuvvIJvfetb+O53v1v09qFQCA899BC2bNmSXnbttddiy5YtaGlpKWeok2pubh63bPny5Vi2bFnWsvvuuw8dHR1Zy5qamioaGxGRldvazs5OtLe3Y9WqVellK1euxI4dOybdV0dHB2655ZYpj9nd3Y3Nmzen2+9Ctik3Jr5EJtfU1IRIJJJ+vWnTJmzatAlutxuhUKjg/Wzfvn1cctnc3Izdu3dj6dKl6Orqwu7du3HgwAFcf/312LdvHwDgwIEDuOqqq7Bnzx60tLSgvb0dhw4dwvbt29He3o4PfOAD+M///E9s2bIF//iP/4hbbrkFiUQCL730Unr99vZ2bN68GYsXL8b+/fvzxnf99denG8x9+/Zh+fLl6O7uxp49ewAAy5YtSyfCu3fvxp49e7Bu3Tpce+212LFjR3rd1DH7+/vT8X/lK18p/AdORDWp2m3tvn37sHbt2pLaWgBZ6xfS1ub7vru7u8e125nH/shHPoLf//73WLp0KbZv356O4W//9m/x9ttvA1Db6AMHDuDQoUPo6upCf38/9uzZg/vuuw/PPPNM1nb5Yi9Xm81SByKTe/PNN7FmzRp4PB4AgMfjwTXXXIO33nqr6H0NDAxM+F5nZyeWL1+OxYsXY/PmzThw4AD27NmDZcuW4Y477sDSpUvTjeGyZcvQ3NyM++67D5/73OfS+1i1ahU6Ojpw1113Za1/++2346qrrsLy5cvR3t6e9/jLly9PJ7kpHR0daGlpQUtLC5599tmsdUOhEJqamtL7y40xM34ioqmYsa3NXb+QtnYiue125rGvvPJKhEKhrJ7iVatW4ZJLLslqo5ctW4b29vb095g6YcjdLl/s5WqzmfgSmdycOXPQ2NiIaDQKt9uNaDSKxsZGzJ49u6j9LFu2LN2zkHLo0CEsX748a1nqUtxVV12F66+/Hg899BBisRh8Ph86OzvTjVJm2cGyZcuwceNGLF26NL0sd/1CrF69Grfffnt6Pxs3boTP58PixYsBIG+vSzAYzHvMzPiL6a0hotpU7bY2VYqgR1ubKRQKpa+mTXbsfDFM1EbnqxueKvZytdlMfIksIBAI4IYbbsBLL72EG264AadOnSp6Hx0dHVi3bl16wMXGjRvx/e9/P/1+KBRKlxBcf/31ePbZZ9Nn/Q8++CA2b96M3bt3o7+/H/v27UNXV1e6cVq9ejW6u7vTjeQ3v/nNrPVvueUWPPvss9i9e3f6Mlg+119/fbonF0BWT0B3dzd27NiBQ4cOpddJ7W/37t249957s46ZGT9rg4moENVsa3/961+X3NbmtntTtbVdXV04dOhQ1uC2VGy57XbmsVPb5caQ20YHg0GEQqH0sVPbFBJ7udpsoVrzVd52223KPffcU9Q2g6NxvPzfR9HU1JQ1eb8gAALUOTcF7TUgQBAAUQAECOqjkHoEREFIvxYzH0UBttTr1HMR2qP62iaqX329PWhrayvnj2XaAoEAY5mAkeIpNZZ33nkHZ5xxRlliMerk40aQG0/uz/3uu+/G/fffX9odRExmOm02AOzY9y6am5vT7bTa9gK5bbQgpNrysXZ6bLmgbYt0259ux7XnY+sIGZ8J2Z8LfX19mNnamrNO9nNxsuUl3jQmk5HaJcBY8RgllnfeeQcLFiwwTNtk9HZST1O12cDE7bahB7cNR5M4eCIMrzHukIeR8AgaGoZhF8V0MmzPeLTbUs/F9Gu7KMBhy3wtwmEbW+awZby2CXBqr8vZ4BIRVYuR2uxwOAyvd7SkfeQm6akEHBhLzFNJOjKS7lSSDu358NAQGhtj6W3HOmSgrSdknRSIU5ws5B5j/EnA2PN8Jx+hYBgnYgM5+xt7jgL2N1Xc2cfO3k7UnggARuMyRmPJsZ9nTueWKPLzkMrH0Imv0ShQkJQVJOVkRY8jQE2SU0mx0ybCYRfhtIlw2tXXI8NDaBtxwJlern65tNcu7TkbDCKi6VOgQFEAKEj9b1rCI3FEhGi5wipZOByGt0/WOwwAqROU8KTrCFqWnHUSgEIS8sx18ifkqZOIM9xJDESSsMeV9FFT74+PZ/wLIc8aQs4b+T6Rx+9fTfgTSRkJJZk+QZgwhqzj54kiZ+PcfQl5Vsr3/SlA+g5pZu6cY+JrQAoUxJMK4smJG6VwuLBeFYeWBKeSYrfDlk6KM1+7HTa47SJcDhFuuw1uh2jqf9hERGQdChRAAZIlnoBMpn2m+rmbNMgdy2VZhigaJBik4hnr+BPyZOR5cvgJkvbss4WJTggcNhH1rvKmqkx8LS6elNUEusiOBgECnHYRbocIj8OmJsYONUH2pB/VZeHRBBpiSSbLRERENSJ1MpK9bNxKE25dCCFvSlwaJr6UlwIF0UQS0UQSA6PxSdcNh8Pwdo9CEAS47SI8ThvqnGpi7HHaUKclznXOsS+PwwannZOKEBERUfUw8aWyURQFo/EkRuNJ9E9ergUAsItiOhH2umyoc9rhcYjwuuzqMqe6rM5pg421ykRERFQiJr6km4QsYzAiYzAyeY+yAAEuhwiv0w6vS02IvS77uEcmyERERDQZXmsmw1OgIBJPoi8cxdH+Ebx5cgj+I0G8+HYvdrxxEtv2H8eTe4/g8RcP4YmXD+Onb/Th5wdP4td/6sEfDgfx3ycGcaRvBH3DMUTjlZ2Rw8y6urrw0Y9+NGtC8+7u7nHLiIho+l4/eACXr7gErx88kF529PDhccuoMtjjS5ahQC21CI/EEembuNYiNUq03mVTH9127bUdDdpyl8NWxcgL43a7StyDun0kkn+kY2dnZ/puQo8++igA9X7yqfuqExHVggVtvrLt60hgYNyy885fjC/f/E/Y8tST2LDxOwCAgcEQ5i9YiPPOX1y2Y1N+THyp5sSTMoIjMQRHJl7HmUqOtaS4MZUcu8eSY7vNehdMfL6JG/zu7m7s2bMHLS0taG9vh8/nw549ewCo91g/dOgQNm7ciFtuuQV79uzBfffdV62wiYhMxdc4cVt79PBhvPzSb9Do82FBezt8jU14+aXfAAAu/tilOHrkMB7/3nfwDzd9Fb998ddYf9c9VYraGpj4EuURS8roH4mhfySW930BAjxOm5oQa8lwQ0aS3Oi2l73XeKKe2kIVervJVatW4YknnsDSpUuzenrvuOMOrFu3DgBw6NAhrFq1CgcOqJflnn32Wdxyyy3YvHkzli9fjmeeeaakWKlyJElaoT39hN/vv1XXYIgMKF8vbSVcfuVnsOXpJ3He4sVZPb0b/uVufPnmr0GRFRw9cgSXf2oxfK83AwB2/Pyn+NJNN+PHT/8QF19yKV742XNVidVKmPgSTYMCBSOxBEZiCWCCG4k4bSKQiOC0VhmNbjUxVh8daHTb4XEar5wCAJYvX45rr70WS5cuHfeez+dDR0cH2tvbsXHjRixevBjt7e04dOgQQqEQmpubdYiYCqUlvVf7/f4vSpJ0qyRJS/x+/3694yKqRRdfcin+4frrcN7i8eUNvsYmzJs/Hwva2/F/v/cIzj3/AsxfsBDHDh/GwEAITU1sa6eLiS9RhcSSMsIjccQmqDd22ES1p9hjR6PbgQa3HT4dE+Ouri5s3rwZ7e3tWLp0Kdrb29HV1ZX+uvfee7F582YsW7YM7e3t6YR3YGAA3d3d2LFjBw4dOoTu7m4cOnQIXV1drA02GL/fvwvALu1lB5Neoup7/eAB/PjpH2L+goW4YPESzF+wEK8fPIA3Dr6G1w8ewPo778GPn/4h/vJjl2Dhwg6cvnAhjh45jIHBARw5fAi7d/4SR48cxtHDh3H0yGG8fvAAa4OLwMSXSCfxKcopnDYRF7YmERqJwyYCNlGAKAiwiUL6eTl1dnZiy5YtAIBbbrklvezNN99Mr5NZt9vR0ZF+vmrVKgDAtddeCwDYsWNHWWOj8pIk6RsAvqh3HES16LzzF+PxzU8BAL50083pZb99dWxGh/V33aPdIljE/IUL08sv/9SnAQCrrl4DAPjxdpY6FIuJL5FBxZIyZFm9g14+mUmwTRAgigLs4thy3j6aJuL3+x+UJGmbJEl+v98fmmi9QCBQ9L4jkUhJsZUTY5mYkeIxSixKqwOKrECGrHcoAGCoWAB94kkmgXieqf6TyezPRVmWC26vmPgSmZSsKJCTCvJNTSxAgKj1Etu0RFiRZUCQIWrJMtUeSZKWAIBW4tANYC2ABydav62trehjuN0n4fV6pxti2TGWiRkpHiPEIghxCKIAUTTGjD0yZMPEAugTj81mm3BQduZyURQLbq9KTnwlSVqrPV3EEcJExqBAQVIGkrKSXibLMsJx9bUgCFlJsU1ExnP2FlvYCgCput4mAK/qGAsRUdWVlLprI4R3+f3+TQA6MqbJISIDUxQFiaSMaCKJkVgCQ5EEQqNx9IVj6BmKoXc4iv5wDIOjcQxHExiNJxFLyEjKChRFmfoAZFSptnotAPj9/u06x0NEVFWl9vh2aF+boF4265h8dSIyunRvMSYooyiytzgUCmHPnj3pAXCkH62ed5PecRBR+Q0MhPDySy+mB8BRfiUlvlpPb8oSAFtLC4eI9PT6wQN4/cABzF+wEADy3hVI7S1WkMjZNjXX5B9ffw03ffVr6QF3Lm8D/uM/tuHTn7mKtcVERCisrZ1Iqq194+Br6VkhAMDna8Lzzz3DxHcKZRncpg2Y2D/VnJDFjhDuG4oZZrQnYJyRpwBjmYyR4ik1FqXVAVkeG0X7w71HS9gZ8PmL5k/49uDAAB79zv/BY088mV72/M+ezTr+ZNv29/fhor/6GBp9PkRi2Wmxp6ERgcFRCILaOwxFgcOeTCfHNgEQRQF6pcWljBAmIuv5t1eOlLyP/3XRgrzLBwZCeOyRh9NTmgEo+A5sAwMhBIP9uPiSS+Frbhr3Pm9sMbVyzeqwopCBbcWOEE44I3C7+w0x2jOFseRnpFgAY8VTSiyCEM8aRVvKoDMFyqQjcnf8/KdY3Lk0a50v3/Q1vHf0KF5+6TeYv2AhBgYH4Gv0jbtP/BsHX8Oxo0fwym9fwoZ/uRsv7HoRAwMh/PvTT+Hc8y/A668dgCiK4+5BPxAMZe3rjn/+Jo4dOYwfP/VDLFm6FKIgoLOzEy/95tdoaWlGR0dHxW6KMd0RwkRExXjhp8/hgsVLspb9w01fTbePqba2oaEB//r972a1ta8fOICjRw7j5Rd/k7etPfhaFwCk9+Vrbsb8BQvGtbXr77oHRw8fxo+f/iHO71RjOe+8C7K2sepNMUqel0KSpLV+v/9B7TkHtxFZyPyFC7HhX+7GtZ/7n7j4kkvx/HPP4OJLLkWjz4eLL7kUgwPqPe3PW7wY8xcsxMWXXIrzL1AT0/u/+c9Y+alP4+JLLk1fztvwL3en1z165Mi4fSVlBd/7zsNY89n/ieWXfQpLPnox1v/v/432D56LplnzcPDNPyM0EsNgJI5wNIFIPIm4Nt8xEZFZ5W1rP1aOtnZB3rYWAB7/3nfw95/7PC7/1Kdx8ccuGbeNVZXU46slug9IknQrgBYAV5clKiKquss//Rnc9k83Zy17+cXfAFAvr/l8TRgcGCj5PvGZ96AH8l+aS13C8/ma0tvMX7gQpy9YgGgif+mFmC6bEMbf6U57j4hIb4W2tYMDAyW3tfMXLkwnw8W0taltrKjUwW27ALCghMgCfL4mfPnmr2HL00+mL7Wdd94FWH/nPXjhp8/B19yMy6/8jHqP+Jz7xA8EQ3jj4GsYGAjh4GtdeP3gAfzDTV/Fjp//FOeef0F6vdx70Ofb1z/c9FU89sjDOL9zCeYvWJDe5q8u+TjmL1iYbqBzpW7okTvoLkUUBNS77PA4bZX7IRIRTaGabW2q3SxnW2t2QrXm5LztttuUe+4pbMRiyvuhCJ5++c+GqdcMh8OMJQ8jxQIYK55SY7l4ZhynzV9YllhS9303Ar1iqXfZ4XWNP9+Px+NZNb7vvPMOzjjjjPTru+++G/fff39NdRlPp80GgPufP2iZv79yMlIsgLHiMUosF8+MY/a8+TXfTk5Ej3hcdhua6sbfuW2qNhuYuN02zk+UiIiIiKiCmPgSERERUU2wTOI72N+Dx9Zdh8H+Hr1DISIiIiIDskziu2vL4zj8x33YteVxvUMhKhsF6p3SqHoURSlpvmQiMi+2t+ZTbJtdrhtY6Gb9lZ1IxGPp13tf2Iq9L2yF3eHEhp916RgZUenCCQGjw0Pw1DcYKhmTkwn0nXgPM+bMg2gzfTOSpigKQqEQXC6X3qEQUZXFZAHJRBw2G2d+MZNoNAq7vfDPIdN/Yq1/ciee/8FDeGPvbsSjEThcbpx70XJc8YV1eodGVLI/hmwAgvAG+0u+nW85ezJHhgYQHQ1jYGAAdQ0+XWMphsshwmkbf6ErNVpZEAS4XC7Mnj276rERkb66hwTY8b5hZlIw2tUnPeKx2wT0OsafiGTOMCEIAmbOnFn4PssWnU4aW2bCVedFIhaF3elCIhaFu64ejS2F/xCIjCquCDgQLM+faTmmDMq9wpJS7BUWvaYvumjRDHxo/vi5KQOBAG9RTFTjemM2HDkaMcTUaoBxpnlL0SOe9lYvrvjg+I6IUtpsY5zWlGg41I8LV67BTQ9vwYUr12Ao2Kd3SESWtP7Jnei89HI4XG4AgMPlRufHL8f6J3fqHBkREdHUTN/jCwDX3flI+vmqG+/UMRIia+MVFiIiMjNL9PiSNXBKOnPgFRYiIjIrS/T4kjVkTkm36sa79A6HJsArLEREZFZMfEl3nJKOiIiIqoGlDqQ7DpgiIiKiamDiS7rjgCnrYJ02EREZGRNfqrp8yREHTFkDbx1ORERGxhpfqrp8g9g4YMrcWKdNRERmwMSXqobJkXXx1uFERGQGLHWgquEgNutinTYREZmB5RNfDrYxDiZH1sY6bSIiMjrLlzrwpgjGkkqOLlx5NX63YxuGgr16h0RlwjptIiIyOssmvqwnNSYmR0RERKQXy5Y6sJ6UiIiIiDJZNvFlPSkRERERZbJs4gtwsA0RERFZEwfvT4/pEt9iftHX3fkIVt14J07rOBurbrwzq76UiIioGEw0yEh4p8zpMV3iy180ERHpgZ8/ZATrr+zEusvOwd4XtkJRFOx9YSvWXXYO1l/ZqXdopmCaWR04SwMREemBnz9kJLxTZmlM0+PLWRqIiEgP/PyxDiuUq3DwfmlMk/jyF01EVDpJktZqXw/oHYtZWPHzZyjYa/oEcDqsUq7CwfvTZ5pSB4B3/SLSw2B/D3604ev47Ppvm/qDngBJklYA2OX3+7slSdomSdIKv9+/S++4zMBqnz8vbttcU3c1tVq5Cm8GNX2mSnz5iyaqPt7221I6tK9NALq151QAq3z+WC0BLBTrYinFVIkvEVVPrX5AWpnf79+U8XIJgK16xUL6SCWAr7+yC4lYtGYSQCuWq9D0MPElorzYQ2JdkiQtAbDf7/fvn2y9QCBQ1H5PnTqFzXfcgL+9ZQMamltLCbEsIpGI3iGkGSUWm6sOotOFRDwGu8OJRCwKm8MNm6sO4XBYl5iq9bMJ9Qaw9JOrIH3iKvh/9SyCPafGfc9G+T0BxooF0Ceeo4MncMmNa/Doo4+ira0tvTwUCk17n0x8iSgv9pBY2gq/33/rVCtlftAU4t5778V7fzqIV557yjBlMV6vt+LHKLQOvhqxFCI6PAjpk6vwV1dem65X1ju2ahz/+nseTT9fdM7Ec97q/bPIZKRYgOrH859PfBevvvoqNm3ahO9+97tZ7xXbPqUw8aWKKsfAKA6u0o/VBvSQOquD3+9/UHtelsFtTU1NWb1BtVYWY7Y6+OvufAThcBher9fU9cpkXbmldps2bcKmTZvgdrtL6u0FTDSdGZlTOaaOscr0M2bE235bizarwwOSJL0rSVKwXPt98803sWbNGng8HgC1M88t76BFVBmpubOd2tzZHo8H11xzDd56662S980eX6qIcgyM4uAqovLSeneby73fOXPmoLGxEdFoNF03WgtlMayDJ6qMVKldPBaF2+1GNBpFY2MjZs+eXfK+2eNLFVGOOx3xbklE5hEIBHDDDTfgCxv+rWYm1LdiHbwV7mxmRbX4exkO9eOTf/P3eOmll3DDDTfg1KlTZdkve3ypIsrxgWDFDxUiq9q6VZ0Z7f7nD2JRDdWNWq0O3mz1ypOx0vgQK/1eCnXdnY+gvdWL88+bjUceKV+ZHRNfqphyfCBY7UOFiKyFN7YwLiski1b8veiNiS9VTDk+EKzyoUJEZGRWqle2UrJopd+LUbDGl4iIqMZZqbTMSuNDrPR7MQr2+BIREZFlSsuslixa5fdiFEx8iYiIyFKlZVZKFq30ezECJr5ERERkKUwWaSKs8SUiIspRi/OmEtUCJr5EREQ5eKv02pI60TFzSQQVhqUOREREGitNhUWFS53ovLhtM9Z87Zt6h0MVxMSXiIhIw3lTa0vuiY5/50/g3/kTnuhYWFlKHSRJWlKO/RAREenJalNh0eRy5/y1O12mnfOXClNy4itJ0goA28oQCxFZFAcKkZmkpsK66eEtuHDlGgwF+/QOqST8+5tY7olOMh7jiY7FlVzq4Pf7d0mS1F2OYIjImjIHCq268S69wyGalNWmwuLf3+Qy5/z97c+2mP5EhyZn6BrfwKmT+OFdX8Tnbn+YZ19EJsSBQkT6mezv745/f1nHyIwl80Tnihtuhdfr1TGa0g329+BHG76Oz67/NnOnPAw9ndn3/s+DOPrmAU4nQ1QFlbgcmls/53C50fnxy3HzY8+V7RhElN9Ef3+sX7U2TsU3uar2+AYCgYLWO/PMMxGNRtOvjXKWGolEdDt2LsYyMSPFY6ZYfvHU93D4j/vwi6e+hyvW3lqWY9pcdRC1AUJ2hxOJWBQ2hxsOTz3C4XBZjlGMYFBEwB0btzwUClU9FqJKm2ygnh5/f1RZvMJWmKomvm1tbQWt99Zbb+G2227DT3/2M0RGR7Omk9H7EoS5oJcvAAAgAElEQVTex8/EWCZmpHiMHkulp/OJDg+m6+d+t2MbhoK9cLvduvxcmpub0dbWlPe9QtsnIjPJrF9N/f2RNXEqvsKUnPhKkrRafZBW+/3+7WWICXPmzEFjYyOikbFeIo6yJKqMSjeW+QYKsbeJqDqsNlCPJsap+ApTjlkdtgMoS8KbKRAI4O+v+zzqz/04Xvv18zxLJaoQNpZERNbAHv6pGXZWh61bt+L9UARPv/xnLOJZKlFFsbEkIjI/9vBPzbCJLxFVDxtLIiKqBYaezqxSeBcbIiIiotpTk4kv57gjIqoedjYQkVHUVKkD57gjIqo+3jKXiIyipnp8eRcbIqLqWX9lJ9Zddg72vrAViqJg7wtbse6yc7D+yk69QyMiA6nmVaGaSnw5bRMRUfWws4GIClHNEtSaKnUAOG0TEVG1sLOBiCajRwlqzSW+nLaJiKh62NlARBPR4zbLNZf4EhFR9bCzgYgmosdVISa+RERERKSLal8VYuJLZTHY34Mfbfg6Prv+26zfIyIiooJU+6pQTc3qQJXDm4IQERGR0bHHl0rCm4IQmY8kSUv8fv9+veMgIqo29vhSSdY/uRPnXvzXnKeTyCQkSVoBYJvecRAR6YGJL5WE83QSmYvf798FoFvvOIhovGrewaxWMfGlkoW1EZk3PbwFF65cg6Fgn94hERERmQ7Hy1Qea3ypZNd840F4vV4AnKeTiIioWBwvUz1MfImIKK9AIFD0NpFIpAKRTI/esQwFe7H94dux+mv3weGp1zWWXHr/bDIxFuDmx57DL596BG/94Tfp0sGzpI/hss9/DeFwWJeYcunxsxl0JBAIjC9OCIVC094nE18iIsqrra2t6G3c7pPpK0BGoGcsv/zhRhx98wBeee4p/PXnbzHUzwXQ92eTq9Zj8Xq9qG/0IRmPwe50IRmPoa6+EbPnLah6LJOp9s+m0eedsB2aTvsEMPElIqopkiStVh+k1X6/f7ve8VgRL1vTdOTewSzYc1LvkCyJiS8RUQ3Rkl0mvBW0/smdeP4HD+GNvbsRj0bgcLlx9l9cis986Ta9QyMDy72DmVFKHKyGszoQERGVUb5pHl0eL6d5JDIAJr5ERERlNpwzzeNwiNM8VgrnvqVisNSBiIiozHjZunoy575ddeNdeodDBsfEl0xpsL8HP9rwdXx2/bd5+ZCIqAZxECFNB0sdyJR4dxsiouoxYjnB+id3ovPSy+FwuQEADpcbnR+/HOuf3KlzZGRk7PElU+EZPhFR9RmxnCDfIEJ3XT2vAtKkmPiSqeSbJujci5bjii+s0zs0IiLLMXpnQ+7ct0PBXr1DIoNj4kumYsUzfNYrE5FRGb2zIXcQIdFUWONLppM7TdBQ0NzTBLFemYiMyoqdDVTb2ONLpmOVM3yjX0IkIgJYTkDWwsTX5HiZ3LyMfgmRiAiwTmcDEcBSB9PjZXLz4iVEIiKi6mKPr0nd+3cX8zK5BfASIhERUfWwx9ekbn7sOU7cbTL5JoC/7s5HsOrGO3Fax9lYdeOdWZcUiYiKYcSbTBAZDRNfk2pobuVlcpNhWQoRVRLbGKKpsdTBxHiZ3Bw4ewMRVRLbmMrhAHLrYY+vifEyuTkY7X7yvBxKZC1Ga2OshL3o1sMeX6IKM9rsDZkN+aob79IlBiIqH6O1MVbAXnTrYuJLVAVGKEsx2kwgvIRIVD5GaGOshPOsWxcTXxPJTBRsrjq9w6EiGGEC+Jsfew57fvyoYRpy9jwTlY8R2hgrYS+6dbHG10SKqTViHSflMspMIOuv7MQ/r/4L7H1hKxRFwd4XtmLdZedg/ZWdVY2DiGgyqV70mx7eggtXrsFQsE/vkKgM2ONrAtOpNWJvGuVjhMuh65/ciWcf34A/vfqiIXqeiYjyYS+6NRk28d27V8BlKxuRVJbAJgKCCAiiAkFQn4vp5wpEERAEbZkICIL2qK1ns4992e0KRDtg116L2jKbQ1vHlvE8tY1DgcOpQIYL9Y0OOJwKHC4FDqcMh0uB0yXDrj13OGXY7Go85ZKv1ujsv7gUn/nSbePXZUE+TcIIDTkvIRIRkV4Mm/gmEkBkVABgQ1zvYIokiKlkWIHDJcPhVF+7PDKcHvUx9TXutVuGq06GK+txFhzuxqxEweXx5k0UWJBPZhA2QM8zERHVHsMmvhddpOCP3QP40X+9A4/HC0UGFFmAogCyLECRAVkGFCX1XH0vtZ6sAEpSgCwDyaSAZFwYe0ykvoBkQkAiIUDWHpOJ8evGYwLiMRGRERlK0qG+jopjj1H1/dSjnBQQHbUhOlrOn8gWiLZ/g8ujQHb24Y+vhLC5/3S462V4vEl46mW465Jw1zdhKLQC8WgCNnsY8WgPRNtceOrbACjlDIho2q75xoPwer0AeAmRiIiqx7CJr80GeOoAl0eGxyvrHQ4AIBwOpz+sJ5NMAvGoiERMQCyVGEdFREfVr5j2GI2IiI6MPcYiY+tER22IjaYSaPV9OelGeAAA5gGYh7f8E0XwVQBfRTKhvtq3S/1yuGR4G5PwNiZQ15jUnidR50uozxuSqPOp76fec7iYLBMREZkdp5BUGTbxNTObDbDVyUAdACTLsk9FAeJRAaPDNoyGbQj2xoCkF6PDIiIjNm25iEjqMayuFxkWMRpW349HRYR6RIR6HAUf1+GS4fWNJcJ1jRmJsS+JuoYk7G4RM+fY0dCcgKdeLmt9cy1jI0VUefw7o1rBQe8qJr4mIQiA063A6U7A15pAQ2sYXm/hvbGpxDk8YEd40IbwoA0jgzaEB+0ID9gylo29Hx7QkuWAE6FAYcexO2Q0NCfUr5acx+YEGlPPm5KwO9mbPBk2UkSVx78zsjoOes/GxLdGjCXOcTTPKmy4oKIA0VERI0O2dHLc+/4IfvvsL/ChC/8GiVgDRgbtCPUJGB10YrDfjuioDcGAE8GAc8r91zWkEuJk3kS5sSUBX2sc7rra6kVmI0VUefw7o1rBQe/ZSk58JUlaDSAEYInf73+w9JDIKAQBcNfJcNfJaNGS5We+/00ET/0HkvE/4G9uUntHMmufYxEBQ0E7BvvtGApqXxnPU8uHg3aMDKlfp45OHofLk4SvVU2Cxx7jaNKeN82MW6rEgo0UUeXx74xqBaeQzFZS4itJ0hIA8Pv9uyRJ6pAkaYnf799fntDISCbrHbnj319OL3e6FcyYE8eMOZP3KssyMDJow2BmYpyTIA/22zHQ60B01IbAMRsCx1wT7s/hktOJsdc3itY5QFNbDM1t8fSX022O0go2UkSVx78zqiVGuHmRUZTa47sGwK+0590AVgBg4mtB5e4dEUWgvimJ+qYk0B6dcD1FAUaHRQz0OrQvO0La40DGY2TEht7jLvQedwHIP/OG15cYS4RnpR7HkmNPvTFmDwHYSBFVA//OqFYY4eZFRlFq4tsEoD/j9YwS90cGNVnvSDgcrthxBQGoa5BR1xDFnEkS5EhYxECfmgQHjssYGfQidMqh1Rs7EOpxqAP7Bux4721P3n2465JoaoujRUuKW+bEMGNODDNmx9EyO1bVHmM2UkSVx78zotpT1cFtgUCBUwNo+oZiiEQiFYqmeLUeS6g3gKWfXAXpE1fB/6tnEew5hXA4bJifS/0M9WvGggjcbnfWe7IMDAcdCPW4MNDrRKjHiYEelzpjRa8LAz1OREZsOHnYhpOH3fn33xRD86zouK+W2VHUN8Xz1hgb5WcDMBYACAZFBNyxcctDoZAO0RARUbWVmviGALRoz5sA9E22cltbW1E7TzgjcLv7C7ppRDkUMp9jtWIpRLVjuf6eR9PPF53TqWssU8kXT0MDMGe+AiCqfQ2l31MUteY4GHAgGHCg/6QD/aec6DvhRP8J9flwSP069qeGcft2uGS0zIqhZXYcM+bE0NDcj67ffAef+tJqnHGeF6JYue+1GEb6PekRS3NzM9ramvK+V2z7RERE5lNq4rsVgKQ97wCwq8T96YrzOdYuQQC8PvWmHPPOHN8bKSeBgT47+k+qyXDfCYf6/KSaGIcH7Th11I1TR1O9xTMAPIof3AbYnTJaT4th5twYZs6LoXVuVH0+N4a6xqRlZqMgIiIyupISX7/fv19SrQAQMuuMDpzPkaYi2oDmtgSa2xJYdP7IuPdHwyL6Tzrw3ZvXQ062AzgTwFkAzkIiNhsnD7vzllB46pOYOTeK1nmxdDI8c14UradVt6aYagenoCSiWlZyja/f799UjkD0xPkcqVQer4y5i6K4/el/1P4tfRfxaAR2pwsf+siV+PAnb8Xo0Cz0HHeh57gTvced6HnPidFhG47+qQ5H/1Q3bp++1jhmztV6iOfF0DYvhrb5UTTNjBumdILMxUhTUPJWwUSkB965DZzPkcon999SMh6Dt1HE2ZIHwGDWuooCDAXVadh6jjszEmIX+k4401O4vfNadi2swyWjbV4UbaerifAs7WvGnBhs/IumyRlmCkqWlhGRHvgxqamF+RzZw1Idmf+WfvuzLRgK5h/zKQhAY0sSjS0j6Dgvu3wimQRCAYeWDLsQOOZE4JgLp465MBy04/i7Hhx/N3taNptdQetcLRFekEqIY2g9LQq7o2LfLplLRaegvOwyB470nQW7wwZRUCCIgGhTIAjq3N2CqODgb38BRYkDuAzAJ7H3hST2vvBbCCJw0eVXQxC17URtOxsgCBnLMh7V91P715aJCkRtnVjcDbfHNbavnHXS24sYizd9DO24NmRvb0PWvsZi0Y6rxZMbr5xUT3ZZ00+kLya+mlqYz5E9LNWR+W/pihtundbsBTYbxu6AJ2XPkzwyJCJwzKUmwkddCBx14tQxF4KnnDh1xI1TR9zAb8fWF0UFrXNjaJ0XxrxFCcxeGMWc9ghaZsUh2qb9bVINKGYKSkUBfv3r0wH4pljzmvzby8B//bzw2MxKEJXsRF7ISMRzkujUupnv5Sba2ScF4/ebN+EXFShyEnaHmHeb8ftNLZ96v+Pisk22X/V5Iu6By20rbL+TxjvZzzH7hGkinPZxYnrEM+hIIBAYX9tXyhSUTHwNqNw9sxy8Zy11DTIWfmgUCz80mrU8OipkJMPq44nDdgRPudKJ8n/vHVvf4ZIxa34UcxZGMGthFHMWRjF7YQQNzZxpwsIqNgWlogA7dsSw5ZVDcLncUGQBsoysR0VW59T+w87n8O5r+yDaXJCTMtrP/Qtc8LHLoWSur0y8vSILkBXtUetJlbX3M7ePRROwiQ7t/ex1Utukth8Xr6IdL5kdi6LtR10HUJIZ+8pZR1G07bV4oQjq+xCAJJCc9q+RSpFKvgVBgWjLTKIBQZQhikL+qwN5riqor3OuCmQl7XlOVkQFyUQUh/97HzrO64TL485O5LWrHMlkHE6XIyu5F0VAyLiKknW8rO3zXeGYYvucqyiZJxKiCESiI6jzesa2y7PO5FdBMq+mFPa7avR5J2yHpjsFJRNfAyp3zywH79UGl0fB6WdFcPpZY2flz3z/m9j7ws9x3sU3YdHitQieaMSJw26cOuzCQJ8D773tGXcnu7rGRDoJntMexWntEcxeGIXDxVkmLKBiU1AKArBsmYI/jAzC6508pXv9v36Kj17eigtXfkorLduKv/zUReUKJS0cDhtm7upwOIy6Oi8G+nrw4w234Zp1D6KhqRVyMifRzk3+kzmJuoyx5D0rCU8l6kI60R+XhMtjifroaAwOhyt/oj6tRD9/vJnfk7r/3O9XQCKehCDY1GXJzHhTJyqZ8WnLJjrZyFh33LLUcbXjAAKSidzfVDUvg12G11+u4uGMRCgsUXc4BFzxPwRs2jTuFzVtTHwNpFI9sxy8V3ty/y29/vIDeP3lB7L+LY0Mieo0a0dcOHHIlX4+MmjHuwftePfgWMIgiApmzo3htI4I5nREcFp7FHM6ImhsSbB32ESMMgVlLZSW5SMIwJ7/9ziOvPl7vLj9Ua1jQ58TSqOdFFQrlrGEODv5Tj2Gh0fgdtflSd6zk/D09gUm6qkrFD/a8A3ISRlqgi1qjzaINic+8+W7spL4SCQGh91VWqKf5+QodWKVN15Z/Z7GTn7G1kskFQgQ0ydX+WIY2z7/SUcyoajft6JdBZHznXxk6+8v77URJr4GUsme2VoYvEdj8v1bOvsvLsVnvnRbep26Bhkd52UPrFMUYKDXjpOH3ThxyIUTh9x4v9uFnvfGyiUOvDhWw+n1JdRkuD2KuYtGMXdRBDPnxVg7bGBWmILSjO79u4tZcmYAggB1wKINGDvpyDj5sCWmvGJRituf/tKEn/ONLdl1q0Y6OQFKi2f9lZ1IJjJvFy8AsMFm9+Cb236fkRxnXrUQMM9Xh5UXlLeTjomvgVSyZ7ZWe1hqVb5/Sy6Pd8p/S4IANM1MoGnmMM7+8HB6eTwm4NQRF97vVhPhE91uvH/IjfCAHW931ePtrvr0ug6XjDntEcxdpH6dtkgrlXCyVIJq182PPYc9P36UJWc1rlavwI7vjHGl//2rN2vK//nQ2ppEue8mz8TXYNgzS5MpZuBj7r+lYM/JaR/X4VQw78xI1u2cFQUIBhx4v1tLiN914/i7boQCThx9qw5H3xq7KYdoU9B2ehRzz4hg3hmjaD0d6PiQwLvTUc1oaG6tyYSHxqvFz3kjJfxMfA2GPbM0mWIGPub+WwqHw5OsXTxBAFpmxdEyK45zPzrWOxwetOH4u2OJ8PvvutHznjN92+Z9u5oAzIEoKmibH8XpZ45i7pkRnH5mBHPaIxxER5ZViwkPjVern/NG+ffPxJcmxBteGIeZpqTzNiZxVmcYZ3WOJdqxiID3D7lx/G033nvHg6N/cqLnPU86GX5Vu5eYKCqYtVBLhs+I4PSzRjGnPcIbcJAl1GrCQwQY598/E1+aMMHlDS+MY6qBj0Y/SXG6FSz84CgWfnAUQBDhcBgOWz3eP+TGe39Wk+H33nYjcEytHz7R7QZ+qW5rs8s4bVEU888axbyzRjH/A6NonRuDOH5OcyIiokkx8aVxCa6ZehdrxVT1UWY8SclNhgG1Z/j4u25tfmH1MXDMhWN/8uDYn8bmG3bXJTHvzAhO/8AoTteSYV9r+eZ5JCIia2LiW8MmSnBtdgc6L72co48NJl99lNVOUpxuBe3njKL9nLG70o2GRbz3Zw+O/dmNY39WE+CBPgfeec2Ld14bm1qncUYc8z8wivlnj2LB2aOYd+YoB88REVEWJr41bLLL57/68WOGGH1JY/LVRw3291j+rnwer4wzO8M4M6NmeKDPribCf/KoyfCfPRjsc+CNVxx445VGAGq98OyFkXQiPP+Do5Db9fouiIjICJj41rDJLp8bZfQlTc5IU8RUk29GAr6PDqdnk5BloPd9J46+6cGRtzw4+pYHJw678X63B+93e/C7Hep2/7dRxp13yPjKVyo3QT0RWZvRx1TQ5Jj41riJElyjjL6kqfEkBRBFoG1eDG3zYpA+MQBArRd+7+2xRPjoWx4M9jtQX8+kl4imz4xjKmgME98axwTX/Pg7zM/pVrJuyawowBnemfjoWY06R0ZEZmS1MRW1ihMCEVFNEARg1hwZzc16R0JEZrT+yZ3ovPRyOFxuAIDD5Ubnxy/H+id36hwZFYOJLxERkYkM9vfgsXXXYbC/R+9QakqtjqmwGia+REREJpJZY0rVlRpTcdPDW3DhyjUYCvbpHRIViTW+REREJsAaU/1xTIX5sceXiIjIBFhjSlQ6Jr5EREQmwBpTotKx1IGIiMgkOG83UWmY+JKlDPb34On7vobP3f4we0GIyHKMVGPKO5iRGbHUgSxl15bHcfTNAxztTERUYZxdgsyIPb5kCRztTERUHXq1t+xhpnJgjy9ZAkc7ExFVh17tLXuYqRzY40uWkDXa2eHkaGciogqp9uwSvKJH5cQeX7KM1GjnL2z4N95Rh4iogqp5BzNe0aNyYo8vWUZqtHM4HMYi3lGHiKhiqjm7BOcvpnJijy9RCQb7e/DYuusw2N+jdyhERJZVzR5msjb2+BKVIHOwxaob79I7HCIiSzLS/MVkbkx8iaaBgy2IiIjMh6UORNPAwRZERETmw8SXaBo42IKIiMh8mPgSTRMHWxBRuXCgLFF1sMaXaJo42IKIyoUDZa2Lt1o2Fia+REREOuFAWevjSY2xMPElIiLSyfond+L5HzyEN/buRjwagcPlxrkXLccVX1ind2hUIp7UGBNrfImIiHTCgbLWxdl/jImJLxFRjZEkaYneMdAYDpS1Jp7UGBNLHYiIaogkSSsA/CuARXrHQioOlLWGfIPYUic1F668Gr/bsQ1DwV6doyQmvkRENcTv9++SJKlb7ziIrCbfIDae1BgPE18iIiKiaeIgNnNh4ktEhsI5L40jEAgUvU0kEqlAJNNjhViGgr3Y/vDtWP21+9DQ3Kp7PJVg9lhufuw5/PKpR/DWH36Truf94Ec+jk9+7isIh8NVjaWS9Ihn0JFAIDB+OFooFJr2Ppn46ogf8ETjcc7L0kiStDbP4m6/37+r2H21tbUVfXy3+yS8Xm/R21WK2WP55Q834uibB/DKc0+V/e/B7D+bSik2Fq/Xi/pGH5LxGOxOF5LxGOobfZg9b0HVY6m0asfT6PNO2A5Np30CmPjqih/wRGN4ubA8/H7/Jr1joNLx78FcOIjNPJj46oANGtF4nMi/OiRJWq0+SKv9fv92veOh/Pj3YC4cxGYeTHx1wAaNaDzOeVkdWrLLhNfg+PdAVBlluYEFJ0MvDhs0ovw4kT/RGP49EJVfyT2+nAx9elgPRDQeLxcSjeHfA1H5lZz4cjL06WGDRkRUXZxJh4jKUupARERkdJkz6RCRejL42LrrMNjfo3coVVPVwW3FTobeNxQz1ATOjCU/I8UCGCsexpKfXrEEgyIC7ti45aVMhk7Gx5l0iPKrxWlVp0x89ZwMPeGMwO3uN9QEzowlPyPFAhgrHsaSnx6xNDc3o62tKe97050MnYyPM+kQZavlk8EpE19Ohj5WF7bq5m8aKnEgosIN9vfgH++6Hs9t+3+YPXu23uFQFXEmHaJstXwyWHKNb+Zk6GWIx5BSlwJe3LZZ71CIaJp2bXkcr/l/j29961t6h0I64NRgRGNq+WSwHLM6WHYy9NxLAf6dP4F/509q4lIAkVXk/h1v2rQJmzZtgtvtZm1vDeFMOkTZanVaVc7qMIn1T+5E56WXw+FyAwDsThc6P3451j+5U+fIiKhQuX/HHo8H11xzDd566y2dIyMi0s91dz6CVTfeidM6zsaqG+/MOjm0Mia+k8i9FJCMx2rmUgCRVWT+HTtdLkSjUTQ2NrLOl4ioBjHxnUJmXdjST65iXVgNq8X5Dq0i9Xe86T924IYbbsCpU6f0DokmwL8zIqqkqs7ja0aZXf9X3HArZ3WoYbU436FVpP6Oz1w0A9f8dW1czjMr/p0RUSUx8SWaQi3Pd0hULfw7I7IWo94inKUORFPIHRzlcLk5yJGozPh3RmQtRr1FOHt8iaZQy/MdElUL/86IrMHoV2/Y40tUAE5+T1R5/DsjMj+jX71hjy9RATj5PVHl8e+MyPyMfvWGiS8RERERlY2R7wrHxJdqglFHlxIREVmNka/esMaXaoJRR5cSERFR9bDHlyzN6KNLiYiIqHrY40uWZvTRpURERFQ9THzJ0ow+upSIiIiqh6UOZHlGHl1KRERE1cPElyzPyKNLiYiIqHpY6kBERERENYGJLxERERHVBCa+RGQKg/09eGzddRjs79E7FCIiMikmvkRkCrwJCRERlYqD26qEt8wlmh7ehISIiMqFPb5Vwt4qounhTUiIiKhc2ONbYeytIioNb0JCRETlwh7fCmNvFVHpUjchuenhLbhw5RoMBfsmXZ8D4YiIKB/2+E5ToTW77K0iKl2xNyHJLC1adeNdlQyNiIhMhD2+01RMzW6xvVVEND3rr+zEusvOwd4XtkJRFOx9YSvWXXYO1l/ZqXdoRDWDV1zIyNjjW6Tp1OzylrlE1bH+yZ14/gcP4Y29uxGPRuBwuXHuRctxxRfW6R2aYUiStFZ7usjv99+qazBkSbziQkbGHt8isWaXyLhYWjQ5SZJWANjl9/s3AejQXhOVRTmvuLDXmCqFiW+R+MFKZGwsLZpUB4BUstutvSYqi3J2DHEKUKoUljpMQ+qD9cKVV+N3O7ZhKNird0hEpGFp0cS0nt6UJQC2TrZ+IBAo+hiRSKTobSqFsUysEvHYXHUQtQ4hu8OJRCwKm8MNm6sO4XC4oFju/buLJywnvOPfXy57zJPFojcjxQLoE8+gI4FAYHwfbSgUmvY+mfhOAz9YicjMJElaAmC/3+/fP9l6bW1tRe/b7T4Jr9c73dDKjrFMrBLxRIcHx3UMFXKc1DqT1elX6+dnpN+TkWIBqh9Po887YTs0nfYJYOJLNA5vL01mljF4LVO33+/flfF6BQe2USWU2jHEckKqNCa+RDk4IpnMLKecYRxJktb6/f4HtecrchJiIt2xnJAqiYkvkYa3lyar02ZxeECSpFsBtAC4WueQiMZhOSFVEhNfIg3ngCWr03p3m/WOg4hIL5zOjEjD2jIiIiJrY48vUQbWlhEREVkXE98y44wA5pHvd8XaMiIiIutiqUOZ8W4z5sHfFRERUW1hj2+ZcEYA8+DvioiIqDaxx7dMynmPcqos/q6sY7C/B4+tuw6D/T16h0JERCbAxLdMOCOAefB3ZR0sVyEiomKw1KGMOCOAefB3ZW4sVyEioulg4ltGnBHAPPi7MjfebISIiKaDpQ5EZDosVyEiqhwrj59g4ktEppQqV7np4S24cOUaDAX79A6JiMgSrDx+gqUORGRKLFchIiqvycZP3PHvL+sYWfmwx5eIiCwldZmWg1aJilML030y8SUiIktJXaZ9cdtmvUMhMpVaGD/BUgciIrKE3Mu0/p0/gX/nTzjNHVERrD7dZ8mJryRJa7Wni/x+/62l7o+IiGg6cvdrYRMAAA7nSURBVKe5sztdOO8vV3CaO6IiWH38REmlDpIkrQCwy+/3bwLQob0mIiIqq0KmV8q9TJuMxyx3mZaISlNqjW8HgFSy2629JiIiKqtCp1fKnOZu6SdXcZo7IspSUqmD1tObsgTA1snWDwQCRe0/HElgjkeG1yMDABQFUAAoijLutQJA1l7Lcuq1AkVRl8uKkv0oj71OKmP7nEwkEikq/kpiLBMzUjyMJT+9YgkGRQTcsXHLQ6GQDtFQIYq9PXXmZdorbrgVXq+3KnES1arB/h78aMPX8dn13zbF1ZWyDG6TJGkJgP1+v3//ZOu1tbUVtd82AD63vejtpkOWFSQVBUl57CuR8zzQ0wtfc4v2WkYiqSAuK0gkZSRkRXutLU+q68QTCuJJGXFZe0zKSMpTJ9mFMFKDbqRYAGPFw1jy0yOW5uZmtLU15X2vGu0MFY+3pyYytsyrMatuvEvvcKY0ZeKbMXgtU7ff79+V8XqF2Qe2iaIAEQIctonXccZdaJtZ+od1Mp0EK4gl1GQ4lpQRS6Qe1fej2utoQntP+4omZERHhZLjICIyulqYXonIjIq9GmMUUya+OeUM40iStNbv9z+oPV+RkxBTHjZRgE20we2Y/j4CgQBmtM5Uk+BEEtG4mhBHMl5HEjKi8SQiCRmjsaT6vva6kNIOIiIjsPr0SkRmZNarMSWVOmizODwgSdKtAFoAXF2WqKggNlFAndOGOuck3dR5KIra0zwaVxPh0XgSkbisPSYxqj0fjSXTj7GkXKHvgohoclafXonIjMx6NabUwW27ADSXKRaqEkEQ4HLY4HLYABTW7ZxIqonyiJYMj8QSGInJGIklcKInCWedByPRJMKxJKKJZGW/ASIiItKdGa/G8M5tVBC7TUSDTUSDe/w/mYBPzhoYlEiqCfJITE2Ew7GE+jyqPh+OqI/RuAx1/g0iIiIyGzNejWHiS2Vnt4lo9Iho9Ezem5yUFYSjCQxHkxiOJtSkWHs+HEloy5KsRyYiIqKyYOJLurGJAho9jkkTZFlWMBJLYiiawFAkgaFoIp0YD0bUZSytICIiokIw8SVDE0UB9W476t12zPHlXyeWkDEUSWAwEsdQJIGjJwHRXY+hSByDkQQicSbGRERExMSXLMBpFzGj3okZ9U4AwGxnNKvmOJaQMRiJY2BUTY4HRxMYSD2OxiGzlIKIiKgmMPEly3PaRbTWu9Ba7xr3nqIoGI4mERqNYyDjKzSiJsUJmdO4ERERWQUTX6ppgiCgwW1Hg9uO05s9We8pioKwlhSHRuMIjYw9sqeYiIjIfJj4Ek1AEMbqi+flJMWyrGQlxP0jMQTD6mvWFBMRjTfY34Mfbfg6Prv+24a/yQFZFxNfomkQRQEtXidavM5x743EkugPxxAciePQ+wpkhwd94ThGYgkdIiUiMoZdWx7H4T/uw64tj2PVjXcBUJPhp+/7Gj53+8NMhqkqmPgSlZl6G2kP5jV7MMsRSQ+0i8ST6A/H0ReOoT8cQ/9IDH3DMYyyh5iILGz9lZ1IxGPp13tf2Iq9L2yF3eHEhz95FY6+eSArGSZjGgr24ql//rLpe+yZ+BJVidthw2lNNpzW5M5aPhJLom84ir5wHL3hKPqGY+gPc2AdEVnD+id34vkfPIQ39u5GPBqBw+VGIh5DIh7D3he2AshOhjf8rEvniCmfF7dtHtdjb0ZMfIl0Vue0oa6lDqe3jC1TFAXBEbV3uHc4hp7hKHqHYgizXIJKJEnSCu3pJ/x+/626BkM1obFlJlx1XiRiUdidLiRiUSxZ9inIyWRWMnzuRctxxRfW6R0u5Zisx96MJylMfIkMSBDGaojPHJuSGKOxJHqGo+gZjqFXS4aDI3Eo4AwTNDUt6b3a7/d/UZKkWyVJWuL3+/frHRdZ33CoHxeuXIMLV16N3+3YhqFgL+qbWtRk2OFEIhaFu67e1JfQrSrVY//6K7uQiEVNf5LCxJfIRDxOG+a31GF+S116WSwhp3uFA0NR9Ayp9cMKp1ujHH6/fxeAXdrLDrMnvZwlwDyuu/OR9PNVN94JAHjqX27GhSvX4IKPX4HXfv08hoK9eoVHk0j12CfjsXSPvZlPUpj4Epmc0y7itCZ3Vu1wPKkmw4GhKE4NqgnxSHhExyjJSCRJ+gaAL061XiAQKHrfkUhkOiFNyy+e+h4O/3EffvHU93DF2vFVG9WMZSpGigUwRjyr/+lbANRY/vrz/wQACIfDeoZkiJ9LipFiCfUGsHjZp/GRy1bD/6tnEew5VZXf1aAjgUBAHB9PKDTtfTLxJbIgh03EHJ8bc3xjyfB7JxxQ3D6cGozi1GAEJwejnGKtRvn9/gclSdomSZLf7/dP+AmSeevvQrndJ+H1ekuKbyq5NYf+nT+Bf+dP8tYcVjqWYhgpFsBY8TCW/IwSy/X3PIpwOAyv14tF53RW7biNPu+E7dB02ieAiS9RzXDaRLQ1e7LuUDcUSeCklgSfGowgMBRFUmaJhJlJkrQ2z+Juv9+/S5KkJQCglTh0A1gL4MFqxlcO+WYJMHPNIRFVDxNfohqm3q65Hme21QMAkrKC3uEoTg5GcWIgghMDEQxH2StsJn6/f9Mkb68AkKrrbQLwauUjKr98swSYueaQiKqHiS8RpdlEAbMa3ZjV6MYF83wA1F7hVBJ8YiCC3jAHzpnYJgB/m+oV9vv923WOZ9ryzRJARDQVJr5ENKlUr/BZs9Re4VhCxomBCN7Xvk4NRlgeYRJaPe9kPcKmkW+WACKiqTDxJaKiOO0iFsyow4IZ6pRqiaSMU0NRvB+K4PhABCcHIognedc5IiIyHia+RFQSu03E3CYP5jZ58GGodcKnBqN4f2AU7wVHcWIgytsvExGRITDxJaKysolCel5haUEzkrKCk4MRvBccxXshtUeYiIhID0x8iaiibKKQ7hH+CNQa4Te6jyMs1uFY/yj6wzHecpmIiKqCiS8RVZXTLmJekwttbTMAACOxJI4FR3GsfwTHgqOcPo2IiCqGiS8R6arOacMHZtXjA9qsEX3DMRwNjuBI/yjeD41yxggiIiobJr5EZCgz6p2YUe9E5+lNSCRlHA9FcLh/BEf7RhAajesdHhERmRgTXyIyLLstY+q0M4HQSBxH+kdwuG8Ex9kbTERERWLiS0Sm0VTnQFOdDxfM8yGWkHEsOIrDfSM42j/C2mAiIpoSE18iMiWnXcSimV4smumFoigIDMVwuC+M7t4R9A5H9Q6vZv39kjbMaJ0JRVHn6lAUZD2Xtdtdp57nW0eBAll7oQCQc9/PWT7R/vr6bWhubp7gOBPvb/xyBVD/yzmOosWbvRyKGn/mOgOIoaHBpR1D/R7Hfd85P6fJlkPbBxEVh4kvEZmeIAiY1ejCrEYXPtLegqFIAoe0JPh4cDSdbFHluewi6pw2vcMAAAScUbS1+fQOAwAQCDjQ1tZW1n2mkurMJBnQXmuJd2bSLmes39Pbi9YZrTnbKlmJeb6Tj1R1kZKT3Bd2XPUYuTEHg4CvqWlcYi8rU514ZJ+cKBnxpOOQx8eWewKUdZISs6HOaU+vn71t6mfCkw4zY+JLRJbT4Lbj/Lk+nD/Xh0g8icN9IzjUNwKHTdA7NKKyEQQBggCIKP7fdSJsx4x6ZwWiKl4gkEhPb6i3QCBQ0AmKkpPkF3PSoW6f/8pB5va9vb1oaZkx4TGmunqS72RgqisJE13pkBUFoVASDY31GDt5meTqTM4JVO5VjtTJRubVkXzfoyiUv81m4ktEluZ22HD27AacPbtB71CIyCJSJx3/v707yGkbCsI4/tFWQuoqIlIPkCNE0xukN6DqCZojtOqKJYIj0BNU5QjcoE/coDkAEjT7Lugiz4pxbRJC4pnw/j8JCRuER/boY/LsgCS93uCFxzoO/x7q3fDtTn72Jm5uDrZ+18LDK+8CAAAAgD4w+AIAAKAIDL4AAAAoAoMvAAAAisDgCwAAgCIw+AIAAKAIDL4AAAAoAoMvAAAAisDgCwAAgCIw+AIAAKAIDL4AAAAoAoMvAAAAisDgCwAAgCIw+AIAAKAIb/o82MnJSZ+HAwA8A5kN4KU5uL+/964BAAAA2DkedQAAAEARGHwBAABQBAbfPWZmX7xrwP/MbNzYPjazicf1aqllmj/OvGup7Xfp45ZzM87X6tijHrx8ZHZMZPb69dT2Rzg3G2V22MHXs/HaeDdfk5lNJH0IUEeYYaHWM1PHGiaSfta2x5KUUrqSNO8KkZ5qmUi6SildSBrlbZdaGvt77+OOer6llC61ODe9XaeXJFJuk9mddZDZD2sgs9esp7G/117eZmaHHHw9G6+jHtfmCy7EsJCPPcs9M/OqpTp+bdcnSfP8+UxSb73TUsuodvxZ3vaqxVWznjwE/MpfO08pXXvVtq8i5TaZ/Sgyu4bMflI9braZ2SEHXzk2XgfX5msys3FuAlcBh4VqZWcUoJbKQNJdbXvoVUhK6SIPApI0lpS8apHi9HH2XtIwr4a5r1buqUi5TWa310Fmr0ZmPyJKL+sZmR118A3TeFLI5jtyPn4lzLCQQ3NmZn/0sHfQkFdWrgP8oonSx5Xb6pxEuA28h8LkNpndiczeQ4EyW4rTy9KGmR118A0pQvMFerVVCTEsmNlAi9WmU0nfzcx1hadmrmVQDCTdOtZSmaSUvnoWELGPtbyNNtdiQMCeI7NbkdmPI7M7BOvljTM76uAbsfGkGM03ym8ImEo6cn7+OdKwMJV0mlI6l/RZUpQVux9a3mYdSXINDTOb5nNUPQfpJVIfS9KlltdpoHw7GE8SMbfJ7IfI7NXI7G6RennjzI46+IZqPClO86WULvMbE6TFxfYUcljI52e+8ht3IK+gWLWSUltZmUia97ny1Kwl13BmZr/z7cXetJwX1z5uqWemxRuyjiUNa7VhfaFym8xuRWY3kNnr1+PZy9vM7LD/sji/ophp8dD7xarv33Et1Z/RuNNiReNjoOV+V/k63Ul6772ykp9Zm0k68u4ZoERRcpvM7kZmo3RhB18AAABgm6I+6gAAAABsFYMvAAAAisDgCwAAgCIw+AIAAKAIDL4AAAAoAoMvAAAAisDgCwAAgCIw+AIAAKAI/wBfs7FvuF6uNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Set into eval mode\n",
    "model.train()\n",
    "model.eval()\n",
    "likelihood.eval()\n",
    "\n",
    "# Initialize plots\n",
    "f, (y1_ax, y2_ax) = plt.subplots(1, 2, figsize=(12, 6))\n",
    "\n",
    "# Make predictions\n",
    "with torch.no_grad(), gpytorch.settings.max_cg_iterations(50):\n",
    "    test_x = torch.linspace(lb, ub, 500)\n",
    "    predictions = likelihood(model(test_x))\n",
    "    mean = predictions.mean\n",
    "    lower, upper = predictions.confidence_region()\n",
    "    \n",
    "# Plot training data as black stars\n",
    "y1_ax.plot(train_x.detach().numpy(), train_y[:, 0].detach().numpy(), 'k*')\n",
    "# Predictive mean as blue line\n",
    "y1_ax.plot(test_x.numpy(), mean[:, 0].numpy(), 'b')\n",
    "# Shade in confidence \n",
    "y1_ax.fill_between(test_x.numpy(), lower[:, 0].numpy(), upper[:, 0].numpy(), alpha=0.5)\n",
    "y1_ax.legend(['Observed Values', 'Mean', 'Confidence'])\n",
    "y1_ax.set_title('Function values')\n",
    "\n",
    "# Plot training data as black stars\n",
    "y2_ax.plot(train_x.detach().numpy(), train_y[:, 1].detach().numpy(), 'k*')\n",
    "# Predictive mean as blue line\n",
    "y2_ax.plot(test_x.numpy(), mean[:, 1].numpy(), 'b')\n",
    "# Shade in confidence \n",
    "y2_ax.fill_between(test_x.numpy(), lower[:, 1].numpy(), upper[:, 1].numpy(), alpha=0.5)\n",
    "y2_ax.legend(['Observed Derivatives', 'Mean', 'Confidence'])\n",
    "y2_ax.set_title('Derivatives')\n",
    "\n",
    "None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
